A nonlinear filter that extends to high dimensional systems

Numerical weather prediction is characterized by high-dimensional, nonlinear systems and poses difficult challenges for real-time data assimilation (updating) and forecasting. The goal of this work is to build on the ensemble Kalman filter (EnsKF) to produce ensemble filtering techniques applicable to non-Gaussian densities in high dimensions. Two filtering algorithms are presented which extend the ensemble Kalman filter by use of Gaussian mixtures. The first method, referred to as a mixture ensemble Kalman filter (XEnsF), adaptively represents local covariance structures using nearest neighbors. An efficient sampling algorithm is presented for XEnsF, and the filter is shown to be superior to existing methods in simulations on a three-dimensional model. A second algorithm, the local-local ensemble filter (LLEnsF), combines localizations in physical as well as phase space, allowing the update step in high dimensional systems to be decomposed into a sequence of lower-dimensional updates tractable by the XEnsF. Given the same ensemble in a 40-dimensional system, the LLensF update is shown to locally produce more accurate estimates of the state than the EnsKF when the underlying distributions are strongly non-Gaussian. In the 40-dimensional system, a hybrid filter combining the output from LLensF with that of EnsKF is shown to outperform the EnsKF by 5.7%.